Video decoding device and video decoding method

ABSTRACT

A video decoding device, in the case where a video of the progressive format is inputted, processes a frame as a picture, in the case where a video of the interlace format is inputted, processes a field as a picture. A video decoding device performs display control corresponding to a format of the both video by analyzing display control information in display control information analyzer. The display control information includes sequence unit display control information which is commonly used in a display process of all pictures that belong to a sequence to be decoded and picture unit display control information which is individually used in a display process of a picture to be decoded. A second code string analyzer acquires each of the sequence unit display control information and the picture unit display control information from an extended information area in units of pictures.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a continuation application of U.S. application Ser. No.14/107,292, filed Dec. 16, 2013, which is a continuation application ofInternational Application No. PCT/JP2012/003910, with an internationalfiling date of Jun. 14, 2012, which claims priorities of Japanese PatentApplication No.: 2011-134891 filed on Jun. 17, 2011, and Japanese PatentApplication No.: 2011-175815 filed on Aug. 11, 2011, the contents ofwhich are incorporated herein by references.

BACKGROUND 1. Technical Field

The present disclosure relates to a video decoding device and a videodecoding method for receiving input encoded progressive video signalsand encoded interlace received video signals and decoding the videosignals.

2. Related Art

In recent years, as multimedia application programs have been developed,integrated use of information from various media including video, audio,and text has prevailed. However, a digitized video contains a largeamount of data. Then, video compression technologies are essential tostore and transmit a video.

On the other hand, standardization of compression technologies is alsoimportant for interoperation of the compressed video data. For example,as standard specifications of video compression technologies, H.261,H.263, and H.264 of ITU-T (International Telecommunication UnionTelecommunication Standardization Sector), MPEG-1, MPEG-2, MPEG-4, andMPEG-4AVC of ISO/IEC (International Organization for Standardization),and the like are known. Further, a standardization activity for thenext-generation video coding system called HEVC (High-Efficiency VideoCoding) is currently carried out by ITU-T jointly with ISO/IEC.

In those kinds of video coding, a picture to be encoded is divided intoencoded unit blocks and redundancy in a time direction and redundancy ina spatial direction are reduced for each of the blocks. Those kinds ofvideo coding compress the amount of information in that manner. In interpredictive coding for reducing time redundancy, a forward or backwardpicture is referenced, so that movement is detected and a predictedimage is created for each block, and then, a difference image betweenthe created predicted image and the block to be encoded is acquired. Inintra predictive coding for reducing spatial redundancy, a predictedimage is generated from pixel information of surrounding encoded blocks,and then, a difference image between the acquired predicted image andthe block to be encoded is acquired. Subsequently, orthogonal transformsuch as discrete cosine transform and quantization are applied to theacquired difference image and then a code string is generated by usingvariable length coding, and as a result, the amount of information iscompressed.

In decoding, the code string which is generated by the above describedencoding process is analyzed, so that predictive information andresidual coefficient information are acquired. Further, inter predictivedecoding and intra-frame predictive decoding are applied by using thepredictive information, so that a predicted image is generated. Then,inverse quantization and inverse orthogonal transform are applied to theresidual coefficient information, so that a difference image isgenerated, and then, the generated predicted image and the generateddifference image are added, so that a final output image is restored.

As two formats of video to be encoded and decoded, a progressive formatand an interlace format are used on this occasion.

In the progressive format, all of the pixels of a screen aresimultaneously imaged to be a frame. On the other hand, in the interlaceformat, a frame is composed of two fields: a top field, which containsonly the pixels of even-numbered lines imaged among the pixels of ascreen; and a bottom field, which contains only the pixels ofodd-numbered lines imaged among the pixels of a screen.

FIGS. 20 and 21 illustrate methods of encoding a progressive video andan interlace video based on H.264, respectively. As illustrated in FIG.20, in encoding a progressive video, a frame is encoded as a picture.For example, Frm3 is encoded as P_Frm1. In the encoding, informationspecifying a display order called POC (Picture Order Count) is assignedto each picture. For example, since P_Frm1 is displayed at the positionof Frm3 in the display order, 3 is assigned to P_Frm1 as the POC.

On the other hand, as illustrated in FIG. 21, a method of encoding afield as a picture and a method of encoding two fields as a picture areknown for encoding the interlace video. For example, FldT6 and FldB7 areencoded as pictures of separate field structures such as P_FldT1 andP_FldB2, whereas FldT12 and FldB13 are encoded as a picture of a singleframe structure such as P_Frm7. A value of the POC is assigned to apicture in the case where a field is encoded as a picture of a fieldstructure, whereas two values of the POC are assigned to a picture inthe case where two fields are encoded as a picture of a frame structure.For example, since P_Frm7 is displayed as divided pictures at twopositions of FldT12 and FldB13, 12 and 13 are assigned to P_Frm7 as thePOCs (ITU-T H.264: Advanced video coding for generic audiovisualservices (March 2010)).

On the other hand, since the HEVC is a coding system which only enablesencoding of a progressive video, only a method of encoding a frame as apicture as illustrated in FIG. 20 is defined in the HEVC (JCT-VC WD2:Working Draft 2 of High-Efficiency Video Coding (March 2011)).

SUMMARY

It is expected that the progressive format will replace the interlaceformat to be the mainstream video format in the future. But on the otherhand, since many of the existing video contents have been created in theinterlace format, effective use of these contents will be also needed inthe future.

However, as described above, the HEVC which is currently in thestandardization process defines an encoding method only for theprogressive format and does not enable encoding of interlace contents.

One non-limiting and exemplary embodiment solves the above describedproblem, and easily realizes supporting of the interlace format inencoding and decoding of a video by using the HEVC.

A video decoding device according to the present disclosure decodes acode string in units of pictures, the code string being acquired as aresult of encoding video signals in an interlace video format or aprogressive video format in units of pictures. The video decoding deviceincludes:

a second code string analyzer that analyzes the code string to acquirean extended information area code string and a first code string;

an extended information area decoder that acquires display controlinformation to be used in displaying a decoded picture, from theextended information area code string;

a picture data decoder that decodes the first code string to acquire thepicture by using common syntax analyzing method and a common signalprocessing method, which are not dependent on the video formats; and

an output picture setter that, in the case where the display controlinformation indicates the progressive format, sets the acquired pictureas a frame and outputs the frame one by one in a display order, and inthe case where the display control information indicates the interlaceformat, sets the acquired picture as a field and outputs a pair of topfield and bottom field in a display order when the pair of top field andbottom field are acquired, wherein

the display control information includes sequence unit display controlinformation and picture unit display control information, the sequenceunit display control information being commonly used in a displayprocess of all pictures that belong to a sequence to be decoded and thepicture unit display control information being individually used in adisplay process of a picture to be decoded, and

the extended information area decoder acquires each of the sequence unitdisplay control information and the picture unit display controlinformation from an extended information area in units of pictures.

A video decoding method according to the present invention decodes acode string in units of pictures, the code string being acquired as aresult of encoding video signals in an interlace video format or aprogressive video format in units of pictures. The video decoding methodincludes:

analyzing the code string to acquire an extended information area codestring and a first code string;

acquiring display control information to be used in displaying a decodedpicture, from the extended information area code string;

decoding the first code string to acquire the picture by using commonsyntax analyzing method and a common signal processing method, which arenot dependent on the video formats; and

in the case where the display control information indicates theprogressive format, sets the acquired picture as a frame and outputs theframe one by one in a display order, and in the case where the displaycontrol information indicates the interlace format, sets the acquiredpicture as a field and outputs a pair of top field and bottom field in adisplay order when the pair of top field and bottom field are acquired,wherein

the display control information includes sequence unit display controlinformation and picture unit display control information, the sequenceunit display control information being commonly used in a displayprocess of all pictures that belong to a sequence to be decoded and thepicture unit display control information being individually used in adisplay process of a picture to be decoded, and

each of the sequence unit display control information and the pictureunit display control information is acquired from an extendedinformation area in units of pictures.

The present disclosure can be realized not only as the above describedvideo decoding device but also as a program or an integrated circuitthat is equivalent to each unit included in the video decoding device.

The video decoding device according to the present disclosure enables adecoding process and display control to be applied to picture data undera common control without increasing the processing amount for both acode string that is acquired as a result of encoding a progressive videoand a code string that is acquired as a result of encoding an interlacevideo. Therefore, the video decoding device facilitates implementationof a decoding device.

Generally, in a video decoding device, a decoding process for decodingan encoded code string to generate a decoded video and a display processfor displaying the generated decoded video on a corresponding displaydevice by adapting the video to the device are controlled as differentlevels. For example, such a method is possible that the former processis famed by hardware and the latter process is famed by software. Themethod can reduce the man-hours for development, for example, indeveloping the devices that are different in the display method such asa television set and a personal computer by using common hardware forperforming the decoding process and only creating software programs forperforming the display process respectively for the devices.

In the present disclosure, the extended information area in that onlythe information unnecessary for the encoding process (display controlinformation, and the like) is described and the other areas in thatinformation necessary for the decoding process is described arecompletely separated. As a result, the present disclosure facilitatessending of only the code strings needed by respective levels of thelevel of performing the decoding process and the level of performing thedisplay process to the respective levels for the purpose like the abovedescribed use, therefore, improves independence of each level. That is,it allows to form the level of performing the decoding process(hardware) by using completely the common components in both thedecoding device that supports the progressive and the decoding devicethat supports the interlace.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of a videoencoding device according to a first embodiment.

FIG. 2 is a flow chart of encoding process according to the firstembodiment.

FIG. 3 is a conceptual diagram for describing an example of a method ofsort in a progressive input video in an encoding order according to thefirst embodiment.

FIG. 4 is a conceptual diagram for describing an example of a method ofsort in an interlace input video in an encoding order according to thefirst embodiment.

FIG. 5 is a conceptual diagram for describing another example of amethod of sort in the interlace input video in an encoding orderaccording to the first embodiment.

FIG. 6 is a conceptual diagram for describing a configuration of a codestring generated by the embodiment.

FIG. 7 is a conceptual diagram for describing an example of a syntax ofa sequence header generated by the embodiment.

FIG. 8 is a conceptual diagram for describing an example of a syntax ofextended information generated by the embodiment.

FIG. 9 is a block diagram illustrating a configuration of a videodecoding device according to a second embodiment.

FIG. 10 is a flow chart of decoding process according to the secondembodiment.

FIG. 11 is a conceptual diagram for describing an example of a method ofsort in a progressive decoded video in a display order according to thesecond embodiment.

FIG. 12 is a conceptual diagram for describing an example of a method ofsort in an interlace decoded video in a display order according to thesecond embodiment.

FIG. 13 is a conceptual diagram for describing another example of amethod of sort in the interlace decoded video in a display orderaccording to the second embodiment.

FIG. 14 is a block diagram illustrating a configuration of a videoencoding device according to a third embodiment.

FIG. 15 is a flow chart of encoding process according to the thirdembodiment.

FIG. 16 is a conceptual diagram for describing a configuration of a codestring generated by the third embodiment.

FIG. 17 is a conceptual diagram for describing an example of a syntax ofextended information generated by the third embodiment.

FIG. 18 is a block diagram illustrating a configuration of a videodecoding device according to a fourth embodiment.

FIG. 19 is a flow chart of decoding process according to the fourthembodiment.

FIG. 20 is a conceptual diagram for describing a conventional encodingmethod of a progressive video.

FIG. 21 is a conceptual diagram for describing a conventional encodingmethod of an interlace video.

DETAILED DESCRIPTION Background of the Embodiments

As described in the section of Problem to be Solved by the Invention,the HEVC which is currently in the standardization process defines anencoding method only for the progressive format and does not enableencoding of interlace contents.

In order to make the HEVC support the interlace format, an approach ofintroducing the same encoding control as that of H.264 which has beendescribed with reference to FIG. 21 can be considered.

However, on the condition that the approach is adopted, the cases wherethe picture to be encoded has a frame structure like P_Frm7, or has atop field structure like P_FldT1, or has a bottom field structure likeP_FldB2 have to be distinguished from each other when the encodingprocess and the decoding process are performed.

Further, in the HEVC, coding information of pictures which have beenencoded once is referenced when the encoding process is performed on thepicture to be encoded. Therefore, for example, in the case where theencoding object is P_Frm7 and the pictures which have their codinginformation referenced are P_FldT1 and P_FldB2, the coding informationof the two pictures which have field structures has to be processed forreference into the coding information of a picture which has a framestructure. That substantially complicates the encoding process.

Then, in the video encoding device of the embodiments, in the case wherevideo format information indicates the progressive format, a frame ofthe video signals is set as a picture, and in the case where the videoformat information indicates the interlace format, a field of the videosignals is set as a picture. Subsequently, pixel data included in theset picture is encoded by using a common signal processing method and acommon syntax structure which are not dependent on video formats.

Further, in the video decoding device of the embodiments, the first codestring is decoded to acquire the picture by using the common syntaxanalyzing method and the common signal processing method, which are notdependent on the video formats. Then, in the case where the displaycontrol information indicates the progressive format, the acquiredpicture is set as a frame and the frames are output one by one in adisplay order, and in the case where the display control informationindicates the interlace format, the acquired picture is set as a fieldand a pair of top field and bottom field are output in a display orderwhen the pair of top field and bottom field are acquired.

The video encoding device and the video decoding device of theembodiments will be described in detail below.

First Embodiment Encoding Process

The first embodiment will be described with reference to the drawings.

1. Configuration of a Video Encoding Device

FIG. 1 is a block diagram of a video encoding device according to theembodiment.

The video encoding device 100 includes a picture setter 101, a videoformat specifying unit 102, a display control information generator 103,a second code string generator 104, and a picture data encoder 110. Thepicture data encoder 110 includes a picture memory 111, a predictionresidual encoder 112, a prediction residual decoder 113, a local buffer114, a prediction encoder 115, a quantization value determiner 116, anda first code string generator 117.

The picture setter 101 sorts input video signals 151 which are input ina display order in units of pictures in an encoding order according tovideo format signals 152 input from the video format specifying unit102, and outputs the sorted input video signals 151 to the picture dataencoder 110. At this moment, in the case where the video format of thepicture is the progressive format, a frame corresponding to the inputvideo signals 151 is set as a picture, and in the case where the videoformat is the interlace format, a field corresponding to the input videosignals 151 is set as a picture.

The picture data encoder 110 divides each picture input from the picturesetter 101 into blocks and performs an encoding process in units ofblocks to generate a code string of picture data. At this moment, acommon encoding process is applied in units of pictures withoutdepending on whether the video format is the progressive format or theinterlace format, and the generated code string of the picture data hasa common syntax.

The video format specifying unit 102 determines whether the video to beencoded is in the progressive format or the interlace format based oninformation specified from outside or information indicated by the inputvideo signals. Then, the video format specifying unit 120 outputs thevideo format signals 152 to the display control information generator103 and the picture setter 101.

The display control information generator 103 generates display controlinformation signals 153 according to the video format signals 152 inputfrom the video format specifying unit 102 and outputs the displaycontrol information signals 153 to the second code string generator 104.

Here, the display control information signals 153 are controlinformation signals which are used in a video decoding devicecorresponding to the video encoding device 100 in displaying a decodedvideo.

The second code string generator 104 encodes the display controlinformation signals 153 output from the display control informationgenerator 103 and information about encoding control in units ofsequences and encoding control in units of pictures as headerinformation to generate a picture upper layer code string. Further, thesecond code string generator 104 associates the generated picture upperlayer code string with the code string of picture data generated by thepicture data encoder 110 to generate code string signals 154 to befinally output.

Now, processes of the picture data encoder 110 will be described.

The picture memory 111 divides an input picture output from the picturesetter 101 in units of pictures into blocks each of which includes aplurality of pixels. A block is a unit of the encoding process. Inresponse to a readout instruction issued from a difference calculator118, the prediction encoder 115, and the quantization value determiner116 in units of blocks resulting from division, corresponding imagesignals are outputted. The block has, for example, horizontal 64 pixelsby vertical 64 pixels, horizontal 32 pixels by vertical 32 pixels, orhorizontal 16 pixels by vertical 16 pixels. That is, the block may be inany size as far as the processing after that is available to the blockin the size.

The prediction residual encoder 112 performs orthogonal transform ondifference image signals 161 output from the difference calculator 118.Further, the prediction residual encoder 112 quantizes an orthogonaltransform coefficient of each of the acquired frequency components andgenerates residual encoding signals 162. Then, the prediction residualencoder 112 outputs the generated residual encoding signals 162 to theprediction residual decoder 113 and the first code string generator 117.At this moment, the prediction residual encoder 112 quantizes theorthogonal transform coefficient by using a quantization valuedetermined by the quantization value determiner 116.

The prediction residual decoder 113 recovers difference imageinformation by performing inverse quantization and inverse orthogonaltransform on the residual encoding signals 162 output from theprediction residual encoder 112. Then, the prediction residual decoder113 outputs generated residual decoding signals 163 to an additioncalculator 119.

The local buffer 114 stores reconstructed image signals 164 output fromthe addition calculator 119. The reconstructed image signals 164 areused as referential pixel data in a predictive encoding process inencoding of pictures after the pictures currently to be encoded. Inresponse to a readout instruction from the prediction encoder 115, thelocal buffer 114 outputs the stored reconstructed image signals 164 tothe prediction encoder 115 as pixel data.

The prediction encoder 115 generates predicted image signals 165 byusing intra prediction or inter prediction based on the image signalsoutput from the picture memory 111. Then, the prediction encoder 115outputs the generated predicted image signals 165 to the differencecalculator 118 and the addition calculator 119. When the predictionencoder 115 uses the inter prediction, it uses the reconstructed imagesignals 164 of already encoded past pictures which have been stored inthe local buffer 114. When the prediction encoder 115 uses the intraprediction, it uses the reconstructed image signals 164 of the currentpictures of an already encoded block adjacent to the block to beencoded. Mode determination on whether to use the intra prediction orthe inter prediction is based on estimation of which of the predictionmethods can reduce the amount of information of residual signals (theamount of information of the residual encoding signals 162, the codestring strings 154, and the like) more.

The quantization value determiner 116 determines the quantization valuefor quantizing the difference image signals 161 in the predictionresidual encoder 112 based on the pictures stored in the picture memory111. As a determination method of the quantization value in thequantization value determiner 116, a so-called rate control-baseddetermination method of the quantization value, by which thequantization value is set to make a bit rate of the code string signals154 approach a target bit rate, may be used.

The first code string generator 117 generates the code string of picturedata by performing variable length coding on the residual encodingsignals 162 output from the prediction residual encoder 112, predictioninformation signals 166 output from the prediction encoder 115, thequantization value output from the quantization value determiner 116,and information about the other encoding control.

The difference calculator 118 generates the difference image signals 161which is a differential value between the image signals read out fromthe picture memory 111 and the predicted image signals 165 output fromthe prediction encoder 115. Then, the difference calculator 118 outputsthe generated difference image signals 161 to the prediction residualencoder 112.

The addition calculator 119 generates the reconstructed image signals164 by adding the residual decoding signals 163 output from theprediction residual decoder 113 and the predicted image signals 165output from the prediction encoder 115. Then, the addition calculator119 outputs the generated reconstructed image signals 164 to the localbuffer 114.

2. Generation Method of Display Control Information

A method of generating the display control information signals 153 inthe display control information generator 103 and describing the displaycontrol information signals 153 in a code string in the second codestring generator 104 and a method of sort in an input video in anencoding order in the picture setter 101 in response to the video formatsignals 152 from the video format specifying unit 102 will be describedspecifically with reference to the flow chart of the entire encodingprocess of FIG. 2.

First, the second code string generator 104 performs code stringgeneration on header areas in units of sequences (S501). Specifically,the second code string generator 104 describes the display controlinformation in units of sequences generated by the display controlinformation generator 103 into header areas in units of sequences in acode string. The display control information generator 103 generatesdisplay control information in units of sequences according to the videoformat specified by the video format specifying unit 102.

Next, the second code string generator 104 performs code stringgeneration on header areas in units of pictures (S502). Specifically,the second code string generator 104 describes the display controlinformation in units of pictures generated by the display controlinformation generator 103 in extended information areas in units ofpictures in a code string. The display control information generator 103generates display control information in units of pictures according tothe video format specified by the video format specifying unit 102.

Next, the picture setter 101 sorts the input picture which is input in adisplay order in an encoding order and selects a picture to be encoded(S503). Specifically, the picture setter 101 sorts the input pictureaccording to the video format specified by the video format specifyingunit 102.

Next, the picture data encoder 110 performs a series of encoding processdescribed with reference to FIG. 1 to generate a code string of picturedata (S504). Meanwhile, in step S504, a common encoding process isapplied without depending on whether the video format is the progressiveformat or the interlace format

Next, when the processes on the currently processed picture to beencoded are completed, the operation returns to step S502 to proceed tothe encoding process of the next picture, and the processes from stepS502 to step S504 are repeated until the encoding processes of all ofthe pictures in the sequence are completed (S505).

3. Sort of Pictures

A sort process of pictures in step S503 will be described in detail withreference to FIGS. 3, 4, and 5.

A method illustrated in FIG. 3 is an example of a sort method ofpictures in the case where the video format is the progressive format.This sort method is completely the same as the conventional process onthe progressive format described with reference to FIG. 20. An inputframe undergoes sort and is encoded as a picture. In the interprediction, only the picture which has been encoded earlier in theencoding order can be referenced. That is, in the case where sort asillustrated in FIG. 3 is performed, P_1 can reference only I_0, and B_2can reference I_0 and P_1. In terms of the display order, since P_1corresponds to Frm3, P_1 can reference only Frm0 in the forwarddirection, and since B_2 corresponds to Frm1, B_2 can reference Frm0 inthe forward direction and Frm3 in the backward direction. With the sortperformed on the encoding order of the pictures as described above, areference method of referential pictures in the inter prediction isallowed to be controlled and more effective prediction becomesavailable. Meanwhile, I_0 is a picture which only undergoes the intraprediction without referencing the other pictures.

On the other hand, methods illustrated in FIGS. 4 and 5 are examples ofa sort method of pictures in the case where the video format is theinterlace format. Unlike the conventional processes on the interlaceformat described with reference to FIG. 21, the methods always performsort and encoding on an input field as a picture. Also a POC which isinformation specifying the display order is always assigned to eachpicture. The encoding processes are performed on each of the pictures,which have undergone the sort, in completely the same method as in thecase where the progressive format is input, without depending on whethereach of the sorted pictures is a top field or a bottom field.

In the example of FIG. 4, the top field and the bottom field belongingto a frame are always paired in the sort. For example, FldT6 and FldB7undergo the sort as P_2 and P_3, so that they are always continual witheach other in the encoding order. That applies to all of the otherpictures. With the above described sort performed, the input top fieldand bottom field are allowed to be continually transferred to theencoding process and a memory management process in the picture setter101 can be simplified.

In contrast, in FIG. 5, the sort is performed regardless of which framethe top field and the bottom field belong to. In a coding structurealone, FIG. 5 is completely in the same structure as that in the case ofthe progressive format described in FIG. 3, but FldT6 and FldB7, forexample, are sorted to be P_4 and B_8 which become apart from each otherby four pictures in the encoding order. The other pictures are alsosorted to be apart from each other by one picture to four pictures inthe encoding order. With the above described sort, the pictures areallowed to undergo the process in completely the same encoding orderboth in the progressive format and the interlace format. However, it isneeded to transfer the input top field and bottom field to the encodingprocess with the fields shifted by maximum of four pictures, whichcomplicates the memory management process in the picture setter 101.

4. Configuration and Syntax of the Code String

Now, a configuration of the code string generated by the embodiment willbe described with reference to FIG. 6.

The code string indicated by the code string signals 154 output from thesecond code string generator 104 includes a sequence header area inwhich encoding control information in units of sequences is described, apicture header area in which encoding control information in units ofpictures is described, an extended information area in which auxiliaryinformation in units of pictures is described, and picture data. Here,the display control information in units of sequences described in stepS501 of FIG. 2 is described in the sequence header area. Further, thedisplay control information in units of pictures described in step S502of FIG. 2 is described in the extended information area. The respectivetypes of information other than the display control information aredescribed in the code string of a common syntax without depending onwhether the video format is the progressive format or the interlaceformat.

Here, the syntax of the sequence header area in which the displaycontrol information in units of sequences is described will be describedin detail with reference to FIG. 7. In FIG. 7, things other than thesyntax related to the embodiment are omitted.

The syntax header is encoded in the syntax configuration inseq_parameter_set_data( ). As the display control information in unitsof sequences, there are three parameters interlace_flag, continual_flag,and max_distance_num. These parameters are described in the area ofvui_parameters( ) in which video information in the sequence headers iscollected.

The parameter interlace_flag specifies whether the video format of theobjective sequence is the progressive format or the interlace format.Specifically, interlace_flag has the value 0 in the case where the videoformat of the objective sequence is the progressive format and has thevalue 1 in the case where the video format of the objective sequence isthe interlace format. However, interlace_flag is not limited to theabove described configuration and may have the value 1 in the case ofthe progressive format and the value 0 in the case of the interlaceformat.

The parameter continual_flag specifies whether the positionalrelationship between the top field and the bottom field belonging to thesame frame is always continual in the encoding order in the case wherethe video format of the objective sequence is the interlace format.Specifically, continual_flag has the value 0 in the case where thepositional relationship between the top field and the bottom fieldbelonging to the same frame is not always continual in the encodingorder, and has the value 1 in the case where the positional relationshipis always continual. That is, continual_flag is the value 1 in the casewhere such sort as described in FIG. 4 is performed, and has the value 0in the case where such sort as described in FIG. 5 is performed. Withthe parameter, the corresponding decoding device is informed whether itis allowed to perform decoding and display control on always continualtwo pictures as a pair. That facilitates determination on possibility ofdecoding. However, continual_flag is not limited to the above describedconfiguration and may have the value 1 in the case where the positionalrelationship between the top field and the bottom field belonging to thesame frame is not always continual in the encoding order, and have thevalue 0 in the case where the positional relationship is alwayscontinual.

The parameter max_distance_num specifies the maximum number of picturesbetween the top field and the bottom field belonging to the same frameapart from each other in the encoding order in the case wherecontinual_flag has the value 0. For example, in the case where such sortas described in FIG. 5 is performed, P_4 and B_8 is the combination bywhich the fields are the most apart from each other and the maximumnumber is four. With the parameter, the decoding device corresponding tothe video encoding device 100 is allowed to determine the number ofpictures which has to be waited for the top field and the bottom fieldbelonging to the same frame to be continual. That facilitatesdetermination of the capacity of the picture memory needed and thedetermination on possibility of decoding.

The parameter names and parameter values described here are merelyexamples and the other parameter names and parameter values may be usedto realize the same functions. Further, not all of the parametersdescribed here but only a part of the parameters may be used. Althoughan example in which each of the above described parameters is describedin the area of vui_parameters( ) has been described here, each of theparameters may be described in any other area as far as the area allowsthe control information in units of sequences to be described.

Now, the syntax of the extended information area in which the displaycontrol information in units of pictures is described will be describedin detail with reference to FIG. 8. In FIG. 8, things other than thesyntax related to the embodiment are omitted.

The extended information area includes a group of parameters called SEI(Supplemental Enhancement Information). The parameter pic_timing_SEI( )of FIG. 8 is one of the SEI and has a function of specifying decodingtiming and display timing of the pictures to be encoded. As the displaycontrol information in units of pictures, there are two parameterspic_struct and field_pair_POC. These parameters are described in thearea of pic_timing_SEI( ).

The parameter pic_struct is a parameter for specifying the displayformat of the pictures to be encoded. The parameter pic_struct has thevalue 0 in the case where the picture is to be displayed as a frame inthe progressive format, the value 1 in the case where the picture is tobe displayed as a top field of the interlace format, the value 2 in thecase where the picture is to be displayed as a bottom field of theinterlace format. In the case where the picture is to be displayed inanother display, the parameter pic_struct has a corresponding valueformat. The parameter pic_struct is not limited to the above describedconfiguration and may have the value 1 in the case where the picture isto be displayed as a frame in the progressive format, the value 0 in thecase where the picture is to be displayed as a top field of theinterlace format, the value 2 in the case where the picture is to bedisplayed as a bottom field of the interlace format. In the case wherethe picture is to be displayed in another display format, the parameterpic_struct has a corresponding value. That is, the parameter pic_structmay have any value as far as the value provides information specifyingthe case where the picture is to be displayed as a frame in theprogressive format, the case where the picture is to be displayed as atop field of the interlace format, and the case where the picture is tobe displayed as a bottom field of the interlace format.

The parameter field_pair_POC is information indicating the POCs of apair of fields belonging to the same frame in the case where pic_structspecifies that the picture to be encoded is to be displayed as a topfield or a bottom field of the interlace format. For example, in FIG. 4,in the case where the picture to be encoded is P_2, the picture to forma pair with P_2 is P_3, therefore, field_pair_POC is 7. Similarly, inFIG. 5, in the case where the picture to be encoded is B_8, the pictureto form a pair with B_8 is P_4, therefore, field_pair_POC is 6. With theparameter, the corresponding decoding device is allowed to determinewhich of the pictures is the top field and the bottom field belonging tothe same frame and, therefore, to perform display control correspondingto the interlace format.

Meanwhile, field_pair_POC may specify the POC value by a differentialvalue of the POC instead of specifying the POC value as it is. Forexample, in FIG. 4, in the case where the picture to be encoded is P_2,the picture to form a pair with P_2 is P_3, therefore, field_pair_POC is7−6=1. Similarly, in FIG. 5, in the case where the picture to be encodedis B_8, the picture to form a pair with B_8 is P_4, therefore,field_pair_POC is 6−7=−1. With the differential value like that, thepicture to from a pair with the picture to be encoded can be specifiedwith a less amount of code.

In the case where continual_flag described in FIG. 7 has the value 1, itis ensured that the top field and the bottom field belonging to the sameframe are in positional relationship of always continual in the encodingorder. Therefore, since the picture to form a pair with the picture tobe encoded can be easily determined, field_pair_POC needs not to bedescribed.

The parameters pic_struct and field_pair_POC which are the displaycontrol information in units of pictures may be described only in thecase where the video format of the objective sequence is the interlaceformat. In that case, pic_struct only needs to be the information whichcan identify the case in which the encoding object is to be displayed asa top field and the case in which the encoding object is to be displayedas a bottom field.

The parameter names and parameter values described here are merelyexamples and the other parameter names and parameter values may be usedto realize the same functions. Further, not all of the parametersdescribed here but only a part of the parameters may be used. Althoughan example in which each of the above described parameters is describedin the area of pic_timing_SEI( ) has been described here, each of theparameters may be described in any other area as far as the area allowsthe control information in units of pictures to be described.

5. Summarization 5-1. Configuration

The video encoding device 100 of the embodiment encodes the input videosignals in units of pictures. The video encoding device 100 includes:

the video format specifying unit 102 (video format information acquiringunit) which acquires video format information indicating whether thevideo format of the video signals is the interlace format or theprogressive format;

the display control information generator 103 which generates displaycontrol information to be used for displaying a video indicated byencoded video signals as a video in a video format indicated by thevideo format information based on the video format information;

the picture setter 101 which, in the case where the video formatinformation indicates the progressive format, sets a frame of the videosignals as a picture and sorts the pictures in the encoding order, andin the case where the video format information indicates the interlaceformat, sets a field of the video signals as a picture and sorts thepictures in the encoding order;

the picture data encoder 110 which generates a block layer code stringby encoding pixel data included in a set picture for each block, whichis a unit of encoding process, by using a common signal processingmethod and a common syntax structure which are not dependent on thevideo formats, further generates a slice layer code string by encodingthe coding control information, which is applied in units of slices inencoding the pixel data, by using the common signal processing methodand the common syntax structure which are not dependent on the videoformats, and outputs a first code string which is acquired by that theblock layer code string is associated with the slice layer code string;

the second code string generator 104 (picture upper layer encoder) whichgenerates the picture upper layer code string by encoding the codingcontrol information which is applied in units of pictures in encodingthe pixel data, the coding control information which is applied in unitsof sequences, and the display control information; and

the second code string generator 104 which outputs a second code string,the second code string being acquired by that the first code string isassociated with the picture upper layer code string.

For example, the display control information generator 103 generatesdisplay control information including sequence unit display controlinformation and picture unit display control information, the sequenceunit display control information being commonly used in the displayprocess of all pictures which belong to a sequence to be encoded and thepicture unit display control information being individually used in thedisplay process of a picture to be encoded.

For example, the second code string generator 104 (picture upper layerencoder) stores the sequence unit display control information in a videoinformation area included in the sequence header which is generated inunits of sequences in the picture upper layer code string.

For example, the second code string generator 104 (picture upper layerencoder) stores the sequence unit display control information of thesame value in each extended information area of each picture belongingto the same sequence in the picture upper layer code string.

For example, the second code string generator 104 (picture upper layerencoder) stores the picture unit display control information in theextended information area which is generated in units of pictures in thepicture upper layer code string.

For example, the display control information generator 103 generates thesequence unit display control information which includes a firstidentifier specifying whether the video signals to be encoded are in theprogressive format or the interlace format.

For example, the display control information generator 103 generates thesequence unit display control information which includes a secondidentifier specifying whether two fields which belong to the same frameare always continual with each other in the encoding order in thesequence in the case where the video signals to be encoded is in theinterlace format.

For example, the display control information generator 103 generates thesequence unit display control information which includes informationindicating the maximum value of an interval between the first field andthe second field which belong to the same frame in the encoding order inthe sequence in the case where the video signals to be encoded are inthe interlace format.

For example, the display control information generator 103 generates thepicture unit display control information which includes a thirdidentifier specifying that the picture to be encoded is (1) to bedisplayed as a frame which belongs to the progressive video signals, (2)to be displayed as a top field which belongs to the interlace videosignals, (3) to be displayed as a bottom field which belongs to theinterlace video signals, or (4) to be displayed in another displayformat in the display process of the picture to be encoded.

For example, the display control information generator 103 generates thepicture unit display control information which includes a thirdidentifier specifying that the picture to be encoded is (1) to bedisplayed as a top field which belongs to the interlace video signals or(2) to be displayed as a bottom field which belongs to the interlacevideo signals in the display process of the picture to be encoded onlyin the case where the video signals to be encoded are in the interlaceformat.

For example, the display control information generator 103 generates thepicture unit display control information which includes specificationinformation specifying a picture that is paired with the picture to beencoded and belongs to the same frame as that of the picture to beencoded in the case where the picture to be encoded is to be decoded anddisplayed as a top field or a bottom field which belongs to theinterlace video signals.

For example, the display control information generator 103 describes, asthe specification information, display order information which isassigned to the picture that is paired with the picture to be encodedand belongs to the same frame as that of the picture to be encoded.

For example, the display control information generator 103 describes, asthe specification information, a difference value between the displayorder information which is assigned to the picture to be encoded and thedisplay order information which is assigned to the picture that ispaired with the picture to be encoded and belongs to the same frame asthat of the picture to be encoded.

The video encoding method of the embodiment encodes the input videosignals in units of pictures. The video encoding method comprising:

acquiring video format information indicating whether the video formatof the video signals is the interlace format or the progressive format,

generating display control information to be used for displaying encodedvideo signals as video signals in the video format indicated by thevideo format information based on the video format information,

in the case where the video format information indicates the progressiveformat, setting a frame of the video signals as a picture and performingsort on the pictures in the encoding order and, on the other hand, inthe case where the video format information indicates the interlaceformat, setting a field of the video signals as a picture and performingsort on the pictures in the encoding order,

generating a block layer code string by encoding pixel data included ina set picture for each block, which is a unit of encoding process, byusing a common signal processing method and a common syntax structurewhich are not dependent on the video formats, further generating a slicelayer code string by encoding the coding control information, which isapplied in units of slices in encoding the pixel data, by using thecommon signal processing method and the common syntax structure whichare not dependent on the video formats, and outputting a first codestring which acquired by that the block layer code string is associatedwith the slice layer code string,

generating the picture upper layer code string by encoding the codingcontrol information which is applied in units of pictures in encodingthe pixel data, the coding control information which is applied in unitsof sequences, and the display control information, and

outputting a second code string which is acquired by that the first codestring is associated with the picture upper layer code string.

5-2. Effects and the Like

With the video encoding device 100 according to the embodiment, anencoding process is allowed to be performed on picture data under acommon control with the processing amount being not increased both inthe case where the progressive video is input and in the case where theinterlace video is input, therefore, the video encoding device 100facilitates implementation of a encoding device.

Second Embodiment Decoding Process

The second embodiment will be described with reference to the drawings.

1. Configuration of a Video Decoding Device

FIG. 9 is a block diagram of a video decoding device according to theembodiment.

The video decoding device 200 includes a second code string analyzer201, a display control information analyzer 202, a picture data decoder210, and an output picture setter 203. The picture data decoder 210includes a first code string analyzer 211, a prediction residual decoder212, a picture memory 213, a prediction decoder 214, and a quantizationvalue determiner 215.

The second code string analyzer 201 extracts display control informationand information about decoding control in units of sequences anddecoding control in units of pictures from a code string of headerinformation included in input code string signals 251. Then, the secondcode string analyzer 201 outputs the extracted information to thedisplay control information analyzer 202 as display control informationsignals 252.

The picture data decoder 210 generates a decoded image of an objectivepicture by performing a decoding process in units of blocks on a codestring of picture data included in the input code string signals 251. Atthis moment, a common decoding process is applied in units of pictureswhether the video format is the progressive format or the interlaceformat and, further, the code string of the picture data to be decodedhas a common syntax.

The display control information analyzer 202 analyzes the displaycontrol information signals 252 output from the second code stringanalyzer 201 to determine whether the video to be decoded is in theprogressive format or the interlace format. Then, the display controlinformation analyzer 202 outputs display control signals 253 based onthe determination result to the output picture setter 203.

The output picture setter 203 sorts the decoded pictures generated bythe picture data decoder 210 in an outputting order according to thedisplay control signals 253 output from the display control informationanalyzer 202. Then, the output picture setter 203 outputs the pictureswhich have undergone the sort to outside as output video signals 254.The output video signals 254 may be in the progressive format and in theinterlace format. In the case where the output video signals 254 are inthe progressive format, display control is performed on a picture as aframe, and in the case where the output video signals 254 are in theinterlace format, the display control is performed on a picture as afield.

Now, processes of the picture data decoder 210 will be described.

The first code string analyzer 211 performs analyzing of decodingcontrol information and analyzing of the picture data in units of blocksby performing variable length decoding on the code string of picturedata of the input code string signals 251. The first code stringanalyzer 211 outputs residual encoding signals 261 which are acquired asa result of analyzing, to the prediction residual decoder 212. Further,the first code string analyzer 211 outputs prediction informationsignals 265 which are acquired as a result of analyzing to theprediction decoder 214. Still further, the first code string analyzer211 outputs quantization value information which is acquired as a resultof analyzing to the quantization value determiner 215.

The prediction residual decoder 212 generates residual decoding signals262 by performing inverse quantization and inverse orthogonal transformon the residual encoding signals 261 output from the first code stringanalyzer 211 and outputs the residual decoding signals 262 to anaddition calculator 216. At this moment, the prediction residual decoder212 performs inverse quantization on the residual encoding signals 261by using a quantization value determined in the quantization valuedeterminer 215.

The picture memory 213 stores reconstructed image signals 263 outputfrom the addition calculator 216. The reconstructed image signals 263are used as referential pixel data in a predictive decoding process indecoding of pictures after the pictures currently to be decoded. Inresponse to a readout instruction from the prediction decoder 214, thepicture memory 213 outputs the stored reconstructed image signals 263 tothe prediction decoder 214 as pixel data. The reconstructed imagesignals 263 are concurrently output to the output picture setter 203 asa final output image.

The prediction decoder 214 generates predicted image signals 264 byusing intra prediction or inter prediction based on the predictioninformation signals 265 output from the first code string analyzer 211and outputs the predicted image signals 264 to the addition calculator216. When the prediction decoder 214 uses the inter prediction, it usesthe reconstructed image signals 263 of already decoded past pictureswhich have been stored in the picture memory 213. When the predictiondecoder 214 uses the intra prediction, it uses the reconstructed imagesignals 263 of the current pictures of an already decoded block adjacentto the block to be decoded. Determination on whether to use the intraprediction or the inter prediction is performed according to the inputprediction information signals 265.

The addition calculator 216 generates the reconstructed image signals263 by adding the residual decoding signals 262 output from theprediction residual decoder 212 and the predicted image signals 264output from the prediction decoder 214 and outputs the reconstructedimage signals 263 to the picture memory 213.

2. Display Control Method

A method of analyzing the display control information in the displaycontrol information analyzer 202 and performing sort on the decodedpictures in an output order in the output picture setter 203 to be anoutput image will be described specifically with reference to the flowchart of the entire decoding process of FIG. 10.

First, the second code string analyzer 201 performs code stringanalyzing of header areas in units of sequences (S1401). At this moment,the display control information analyzer 202 acquires the displaycontrol information in units of sequences.

Next, the second code string analyzer 201 performs code string analyzingof header areas in units of pictures (S1402). At this moment, thedisplay control information analyzer 202 acquires the display controlinformation in units of pictures.

Next, the picture data decoder 210 performs a series of decoding processdescribed with reference to FIG. 9 to generate a decoded image of theobjective pictures (S1403). Meanwhile, in step S1403, a common decodingprocess is applied without depending on whether the video format is theprogressive format or the interlace format.

Next, the output picture setter 203 sorts the decoded image which isstored in a decoding order in a display order and a process of selectinga picture to be displayed (S1404). At this moment, the output picturesetter 203 sorts the pictures according to a display control methodacquired as a result of analyzing by the display control informationanalyzer 202.

Next, when the processes on the currently processed picture to bedecoded are completed, the operation returns to step S1402 to proceed tothe decoding process of the next picture, and the processes from stepS1402 to step S1404 are repeated until the decoding processes of all ofthe pictures in the sequence are completed (S1405).

3. Sort of Pictures

A sort process of pictures in step S1404 will be described in detailwith reference to FIGS. 11, 12, and 13.

A method illustrated in FIG. 11 is an example of a sort method ofpictures in the case where the video format is the progressive format.This sort method is the reverse of the sort process for the progressiveformat performed in the encoding device described with reference to FIG.3, and in the sort method, each of the decoded pictures is always sortedas a frame and displayed.

On the other hand, methods illustrated in FIGS. 12 and 13 are examplesof a sort method of pictures in the case where the video format is theinterlace format. These sort methods are the reverse of the sortprocesses for the interlace format performed in the encoding devicedescribed with reference to FIGS. 4 and 5, and in the sort methods, eachof the decoded pictures is always sorted as a field and displayed.

In FIG. 12, the top field and the bottom field belonging to a frame arealways paired in the sort. For example, P_2 and P_3 undergo the sort asFldT6 and FldB7, so that they are always continual with each other inthe display order. That applies to all of the other pictures. With theabove described sort performed, the decoded top field and bottom fieldare allowed to be continually transferred to the display process and amemory management process in the output picture setter 203 can besimplified.

In contrast, in FIG. 13, the sort is performed regardless of which framethe top field and the bottom field belong to. In a coding structurealone, FIG. 13 is completely in the same structure as that in the caseof the progressive format described in FIG. 11, but P_4 and B_8, forexample, are sorted to be FldT6 and FldB7, therefore, the two fieldswhich have apart from each other by four pictures in the decoding orderneed to undergo the sort to be paired. Also for the other pictures, thetwo pictures which have become apart from each other by one picture tofour pictures in the decoding order have undergone the sort to bepaired. With the above described sort, the pictures are allowed toundergo the process in completely the same decoding order both in theprogressive format and the interlace format. However, it is needed totransfer the decoded top field and bottom field to the display processwith the fields shifted by maximum of four pictures, which complicatesthe memory management process in the output picture setter 203.

In the picture memory 213, for the purpose of limiting the memorycapacity, a process for ensuring an area for pictures to be stored isperformed in such a manner as deleting a picture which has becomeunnecessary among the stored pictures from the memory.

At this moment, the picture which meets two conditions that (1) thepicture is no longer used as referential pixel data in the predictiveencoding process and that (2) the picture has already been transferredto the display process can be deleted. That is, in the case where thevideo format is the interlace format, when the decoding process on thepictures of both the top field and the bottom field belonging to a framehas completed and the display process on both of the pictures hascompleted, the pictures can be deleted from the picture memory 213.

4. Configuration and Syntax of the Code String

Since the configuration and syntax of the code string to be decoded inthe embodiment are completely the same as those described in the firstembodiment with reference to FIGS. 6 to 8, the description of them willbe omitted here.

5. Summarization 5-1. Configuration

A video decoding device 200 of the embodiment decodes a code string inunits of pictures, the code string being acquired as a result ofencoding video signals in an interlace video format or a progressivevideo format in units of pictures. The video decoding device 200includes:

the second code string analyzer 201 which analyzes the code string toacquire a picture upper layer code string and a first code string;

the second code string analyzer 201 (picture upper layer decoder) whichacquires, from the picture upper layer code string, coding controlinformation used in encoding the video signals and display controlinformation to be used for displaying decoded pictures;

the picture data decoder 210 which decodes the first code string foreach block, which is a unit of decoding process, to acquire the pictureby using common syntax analyzing method, a common signal processingmethod, and information commonly used in encoding regardless of thevideo format among the coding control information, which are notdependent on the video formats; and

the output picture setter 203 which, in the case where the displaycontrol information indicates the progressive format, sets the acquiredpicture as a frame and outputs the frame one by one in a display order,and in the case where the display control information indicates theinterlace format, sets the acquired picture as a field and outputs apair of top field and bottom field in a display order when the pair oftop field and bottom field are acquired.

For example, the second code string analyzer 201 (picture upper layerdecoder) includes sequence unit display control information and pictureunit display control information, the sequence unit display controlinformation being commonly used in a display process of all pictureswhich belong to a sequence to be decoded and the picture unit displaycontrol information being individually used in a display process of apicture to be decoded.

For example, the second code string analyzer 201 (picture upper layerdecoder) acquires the sequence unit display control information in avideo information area included in the sequence header which is analyzedin units of sequences in the picture upper layer code string.

For example, each extended information area of each picture unit whichbelongs to the same sequence stores the sequence unit display controlinformation of the same value.

For example, the second code string analyzer 201 (picture upper layerdecoder) acquires the picture unit display control information from theextended information area which is analyzed in units of pictures in thepicture upper layer code string.

For example, the second code string analyzer 201 (picture upper layerdecoder) acquires the sequence unit display control information whichincludes a first identifier specifying whether the video signals to bedecoded are in the progressive format or the interlace format, and

the output picture setter 203 changes an output method of the picturebased on the acquired first identifier.

For example, in the case where the video signals to be decoded are inthe interlace format, the second code string analyzer 201 (picture upperlayer decoder) acquires the sequence unit display control informationwhich includes a second identifier specifying whether two fields whichbelong to the same frame are always continual with each other in thedecoding order in the sequence, and the output picture setter 203changes the output method of the picture based on the acquired secondidentifier.

For example, in the case where the video signals to be decoded are inthe interlace format, the second code string analyzer 201 (picture upperlayer decoder) acquires the sequence unit display control informationwhich includes information indicating the maximum value of an intervalbetween the first field and the second field which belong to the sameframe in the decoding order in the sequence, and

the output picture setter 203 changes the output method of the picturebased on the acquired information indicating the maximum value.

For example, the second code string analyzer 201 (picture upper layerdecoder) acquires the picture unit display control information whichincludes a third identifier specifying that the picture to be decoded is(1) to be displayed as a frame which belongs to the progressive videosignals, (2) to be displayed as a top field which belongs to theinterlace video signals, (3) to be displayed as a bottom field whichbelongs to the interlace video signals, or (4) to be displayed inanother display format in the display process of the picture to bedecoded, and the output picture setter 203 changes the output method ofthe picture based on the acquired third identifier.

For example, the second code string analyzer 201 (picture upper layerdecoder) acquires the picture unit display control information whichincludes a third identifier specifying that the picture to be decoded is(1) to be displayed as a top field which belongs to the interlace videosignals or (2) to be displayed as a bottom field which belongs to theinterlace video signals in the display process of the picture to bedecoded only in the case where the video signals to be decoded are inthe interlace format, and

the output picture setter 203 changes the output method of the picturebased on the acquired third identifier.

For example, the second code string analyzer 201 (picture upper layerdecoder) acquires the picture unit display control information whichincludes specification information specifying a picture that is pairedwith the picture to be decoded and belongs to the same frame as that ofthe picture to be decoded in the case where the picture to be decoded isto be displayed as a top field or a bottom field which belongs to theinterlace video signals, and

the output picture setter 203 outputs the picture to be decoded bysetting the picture to be decoded as paired with the picture specifiedin the specification information.

For example, the second code string analyzer 201 (picture upper layerdecoder) acquires, as the specification information, display orderinformation which is assigned to the picture that is paired with thepicture to be decoded and belongs to the same frame as that of thepicture to be decoded.

For example, the second code string analyzer 201 (picture upper layerdecoder) acquires, as the specification information, a difference valuebetween the display order information which is assigned to the pictureto be decoded and the display order information which is assigned to thepicture that is paired with the picture to be decoded and belongs to thesame frame as that of the picture to be decoded.

A video decoding method of the embodiment decodes a code string in unitsof pictures, the code string being acquired as a result of encodingvideo signals in an interlace video format or a progressive video formatin units of pictures. The video decoding method comprising:

analyzing the code string to acquire a picture upper layer code stringand a first code string,

acquiring, from the picture upper layer code string, coding controlinformation used in encoding the video signals and display controlinformation to be used for displaying decoded pictures,

decoding the first code string for each block, which is a unit ofdecoding process, to acquire the picture by using common syntaxanalyzing method, a common signal processing method, and informationcommonly used in encoding regardless of the video format among thecoding control information, which are not dependent on the videoformats, and

in the case where the display control information indicates theprogressive format, setting the acquired picture as a frame andoutputting the frame one by one in a display order, and in the casewhere the display control information indicates the interlace format,setting the acquired picture as a field and outputting a pair of topfield and bottom field in a display order when the pair of top field andbottom field are acquired.

5-2. Effects and the Like

With the video decoding device 200 according to the embodiment, adecoding process and display control are allowed to be performed onpicture data under a common control with the processing amount being notincreased for both a code string which is acquired as a result ofencoding a progressive video and a code string which is acquired as aresult of encoding an interlace video. That facilitates implementationof a decoding device.

Third Embodiment Another Example of Encoding Process

The third embodiment will be described with reference to the drawings.

1. Configuration of a Video Encoding Device

FIG. 14 is a block diagram of a video encoding device according to theembodiment.

The video encoding device 100-1 includes a picture data encoder 110-1and a second code string generator 104-1 in place of the picture dataencoder 110 and the second code string generator 104 of the videoencoding device 100 of the first embodiment. The picture data encoder110-1 includes a first code string generator 117-1 in place of the firstcode string generator 117.

For convenience of the description, a detailed description of the sameconfiguration as that of the first embodiment will be omitted below.Further, in FIG. 14, blocks which have the same functions as those ofthe blocks in FIG. 1 are denoted by the same numbers as the blocks inFIG. 1.

The picture data encoder 110-1 divides each picture input from thepicture setter 101 into blocks and performs an encoding process in unitsof blocks to generate a code string of picture data. At this moment, acommon encoding process is applied in units of pictures withoutdepending on whether the video format is the progressive format or theinterlace format and, the generated code string of the picture data hasa common syntax. Also in the embodiment, the picture data encoder 110-1generates a code string of sequence header and a code string of pictureheader.

The first code string generator 117-1 generates the code string ofpicture data by performing variable length coding on the residualencoding signals 162 output from the prediction residual encoder 112,prediction information signals 166 output from the prediction encoder115, the quantization value output from the quantization valuedeterminer 116, and information about the other encoding control. In theembodiment, the first code string generator 117-1 encodes informationabout encoding control in units of sequences and encoding control inunits of pictures as header information to generate the code string ofsequence header and the code string of picture header.

The second code string generator 104-1 encodes the display controlinformation signals 153 output from the display control informationgenerator 103 to generate an extended information area code string. Thedisplay control information signals 153 include the display controlinformation in units of sequences and the display control information inunits of pictures. Further, the second code string generator 104-1associates the generated extended information area code string and thecode string generated by the picture data encoder 110, the code stringof picture data including a sequence header, a picture header, andpicture data, to generate the code string signals 154 to be finallyoutput.

2. Generation Method of Display Control Information

A method of generating the display control information signals 153 inthe display control information generator 103 and describing the displaycontrol information signals 153 in a code string in the second codestring generator 104-1 and a method of sort in an input video in anencoding order in the picture setter 101 when the video format signals152 are received from the video format specifying unit 102 will bedescribed specifically with reference to the flow chart of the entireencoding process of FIG. 15.

First, the first code string generator 117-1 performs code stringgeneration on header areas in units of sequences (S1801).

Next, the first code string generator 117-1 performs code stringgeneration on header areas in units of pictures (S1802).

Next, the display control information generator 103 generates displaycontrol information in units of sequences and display controlinformation in units of pictures according to the video format specifiedby the video format specifying unit 102 (S1803). The second code stringgenerator 104-1 encodes and describes the respective kinds of displaycontrol information in the extended information area in the code string.

Next, the picture setter 101 sorts the input picture which is input in adisplay order in an encoding order and selects a picture to be encoded(S1804). At this moment, the picture setter 101 sorts the picturesaccording to the video format specified by the video format specifyingunit 102.

Next, the picture data encoder 110-1 performs the above described seriesof encoding process to generate a code string of picture data (S1805).Meanwhile, in step S1805, a common encoding process is applied withoutdepending on whether the video format is the progressive format or theinterlace format.

Next, when the processes on the currently processed picture to beencoded are completed, the operation returns to step S1802 to proceed tothe encoding process of the next picture, and the processes from stepS1802 to step S1805 are repeated until the encoding processes of all ofthe pictures in the sequence are completed (S1806).

3. Sort of Pictures

As for the sort process of pictures in step S1804, the same processes asthose described with reference to FIGS. 3, 4, and 5 in the firstembodiment are performed. Therefore, the description of the process willbe omitted.

4. Configuration and Syntax of the Code String

Now, a configuration of the code string generated by the embodiment willbe described with reference to FIG. 16.

The generated code string includes a sequence header area in whichcoding control information in units of sequences is described, a pictureheader area in which coding control information in units of pictures isdescribed, an extended information area in which auxiliary informationin units of pictures is described, and picture data. Both of the displaycontrol information in units of sequences and the display controlinformation in units of pictures are described in the extendedinformation area. The respective types of information other than thedisplay control information are described in the code string of a commonsyntax without depending on whether the input video format is theprogressive format or the interlace format.

Now, the syntax of the extended information area in which the displaycontrol information in units of sequences and the display controlinformation in units of pictures are described will be described indetail with reference to FIG. 17. In FIG. 17, things other than thesyntax related to the embodiment are omitted.

In FIG. 17, all of interlace_flag, continual_flag, max_distance_flag,pic_struct, and field_pair_POC are described in pic_timing_SEI( ) Sincedetails of the parameters are completely the same as those described inFIGS. 7 and 8, the descriptions will be omitted here.

Since the extended information area is encoded for each of the pictures,as for interlace_flag, continual_flag, and max_distance_flag of thedisplay control information in units of sequences, the same values arealways repeated for each picture in the sequence although it isredundant. However, with all of the display control informationdescribed collectively as illustrated in FIG. 17, all of the syntaxareas other than the extended information area may be in a common syntaxwithout depending on whether the video format is the progressive formator the interlace format. Therefore, the encoding process can be furthersimplified.

The parameter names and parameter values described here are merelyexamples and the other parameter names and parameter values may be usedto realize the same functions. Further, not all of the parametersdescribed here but only a part of the parameters may be used. Althoughan example in which each of the above described parameters is describedin the area of pic_timing_SEI( ) has been described here, each of theparameters may be described in any other area as far as the area allowsthe control information in units of pictures to be described.

5. Summarization 5-1. Configuration

The video encoding device 100-1 of the embodiment encodes the inputvideo signals in units of pictures. The video encoding device 100-1includes:

the video format specifying unit 102 (video format information acquiringunit) which acquires video format information indicating whether thevideo format of the video signals is the interlace format or theprogressive format;

the display control information generator 103 which generates displaycontrol information to be used for displaying a video indicated byencoded video signals as a video in a video format indicated by thevideo format information based on the video format information;

the picture setter 101 which, in the case where the video formatinformation indicates the progressive format, sets a frame of the videosignals as a picture and sorts the pictures in the encoding order, andin the case where the video format information indicates the interlaceformat, sets a field of the video signals as a picture and sorts thepictures in the encoding order;

the picture data encoder 110-1 which outputs a first code string byencoding the pixel data included in the set picture by using a commonsignal processing method and a common syntax structure which are notdependent on the video formats;

the second code string generator 104-1 (extended information areaencoder) encodes the display control information to generate theextended information area code string; and

the second code string generator 104-1 which outputs a second codestring, the second code string being acquired by that the first codestring is associated with the extended information area code string,wherein

the display control information includes sequence unit display controlinformation and picture unit display control information, the sequenceunit display control information being commonly used in a displayprocess of all pictures which belong to a sequence to be encoded and thepicture unit display control information being individually used in adisplay process of a picture to be encoded, and

the second code string generator 104-1 (extended information areaencoder) stores the extended information area code string in theextended information area which is generated in units of pictures.

For example, the second code string generator 104-1 (extendedinformation area encoder) stores the sequence unit display controlinformation of the same value in each extended information area of eachpicture belonging to the same sequence.

For example, the display control information generator 103 generates thesequence unit display control information which includes a firstidentifier specifying whether the video signals to be encoded are in theprogressive format or the interlace format.

For example, the display control information generator 103 generates thesequence unit display control information which includes a secondidentifier specifying whether two fields which belong to the same frameare always continual with each other in the encoding order in thesequence in the case where the video signals to be encoded is in theinterlace format.

For example, the display control information generator 103 generates thesequence unit display control information which includes informationindicating the maximum value of an interval between the first field andthe second field which belong to the same frame in the encoding order inthe sequence in the case where the video signals to be encoded are inthe interlace format.

For example, the display control information generator 103 generates thepicture unit display control information which includes a thirdidentifier specifying that the picture to be encoded is (1) to bedisplayed as a frame which belongs to the progressive video signals, (2)to be displayed as a top field which belongs to the interlace videosignals, (3) to be displayed as a bottom field which belongs to theinterlace video signals, or (4) to be displayed in another displayformat, in the display process of the picture to be encoded.

For example, the display control information generator 103 generates thepicture unit display control information which includes a thirdidentifier specifying that the picture to be encoded is (1) to bedisplayed as a top field which belongs to the interlace video signals or(2) to be displayed as a bottom field which belongs to the interlacevideo signals in the display process of the picture to be encoded onlyin the case where the video signals to be encoded are in the interlaceformat.

For example, the display control information generator 103 generates thepicture unit display control information which includes specificationinformation specifying a picture that is paired with the picture to beencoded and belongs to the same frame as that of the picture to beencoded in the case where the picture to be encoded is to be decoded anddisplayed as a top field or a bottom field which belongs to theinterlace video signals.

For example, the display control information generator 103 describes, asthe specification information, display order information which isassigned to the picture that is paired with the picture to be encodedand belongs to the same frame as that of the picture to be encoded.

For example, the display control information generator 103 describes, asthe specification information, a difference value between the displayorder information which is assigned to the picture to be encoded and thedisplay order information which is assigned to the picture that ispaired with the picture to be encoded and belongs to the same frame asthat of the picture to be encoded.

The video encoding method of the embodiment encodes the input videosignals in units of pictures. The video encoding method comprises:

acquiring video format information indicating whether the video formatof the video signals is the interlace format or the progressive format,

generating display control information to be used for displaying thevideo indicated by the encoded video signals as a video in the videoformat indicated by the video format information based on the videoformat information,

in the case where the video format information indicates the progressiveformat, performing sort on a plurality of frames included in the videosignals in the encoding order and also setting each of the plurality offrames as a picture, and in the case where the video format informationindicates the interlace format, performing sort on a plurality of fieldsincluded in the video signals in the encoding order and also settingeach of the plurality of fields as a picture,

outputting a first code string by encoding the pixel data included inthe set picture by using a common signal processing method and a commonsyntax structure which are not dependent on the video formats,

encoding the display control information to generate the extendedinformation area code string,

-   -   outputting a second code string which is acquired by that the        first code string is associated with the extended information        area code string, wherein

the display control information includes sequence unit display controlinformation and picture unit display control information, the sequenceunit display control information being commonly used in a displayprocess of all pictures which belong to a sequence to be encoded and thepicture unit display control information being individually used in adisplay process of a picture to be encoded, and

storing each of the sequence unit display control information and thepicture unit display control information in the extended informationarea which is generated in units of pictures.

5-2. Effects and the Like

With the video encoding device 100-1 according to the embodiment, anencoding process is allowed to be performed on picture data under acommon control with the processing amount being not increased both inthe case where the progressive video is input and in the case where theinterlace video is input, therefore, the video encoding device 100-1facilitates implementation of a encoding device.

Generally, in a video decoding device, a decoding process for decodingan encoded code string to generate a decoded image and a display processfor displaying the generated decoded image on a corresponding displaydevice by adapting the image to the device are controlled as differentlevels. For example, such a method is possible that the former processis formed by hardware and the latter process is famed by software. Themethod can reduce the man-hours for development, for example, indeveloping the devices which are different in the display method such asa television set and a personal computer, by using common hardware forperforming the decoding process and only creating software programs forperforming the display process respectively for the devices.

In the embodiment, the extended information area in which only theinformation unnecessary for the encoding process (display controlinformation, and the like) is described and the other areas in whichinformation necessary for the decoding process is described arecompletely separated as in the code string described in FIG. 16. As aresult, the present disclosure facilitates sending of only the codestrings needed by respective levels of the level of performing thedecoding process and the level of performing the display process to therespective levels for the purpose like the above described use,therefore, improves independence of each level. That is, it allows toform the level of performing the decoding process (hardware) by usingcompletely the common components in both the decoding device whichsupports the progressive format and the decoding device which supportsthe interlace format.

Fourth Embodiment Another Example of Decoding Process

The fourth embodiment will be described with reference to the drawings.

1. Configuration of a Video Decoding Device

FIG. 18 is a block diagram of a video decoding device according to theembodiment.

The video decoding device 200-1 includes a picture data decoder 210-1and a second code string analyzer 201-1 in place of the picture datadecoder 210 and the second code string analyzer 201 of the videodecoding device 200 of the second embodiment. The picture data decoder210-1 includes a first code string analyzer 211-1 in place of the firstcode string analyzer 211.

For convenience of the description, a detailed description of the sameconfiguration as that of the second embodiment will be omitted below.Further, in FIG. 18, blocks which have the same functions as those ofthe blocks in FIG. 1 are denoted by the same numbers as the blocks inFIG. 1.

The second code string analyzer 201 extracts at least display controlinformation from the extended information area of the code string ofheader information included in the input code string signals 251. Then,the second code string analyzer 201 outputs the extracted information tothe display control information analyzer 202 as display controlinformation signals 252.

The picture data decoder 210 generates a decoded image of an objectivepicture by performing a decoding process in units of blocks on a codestring of picture data included in the input code string signals 251. Atthis moment, a common decoding process is applied in units of pictureswithout depending on whether the video format is the progressive formator the interlace format and, further, the code string of the picturedata to be decoded has a common syntax.

The first code string analyzer 211 performs analyzing of decodingcontrol information and analyzing of the picture data in units of blocksby performing variable length decoding on the code string of picturedata of the input code string signals 251. The decoding controlinformation includes information about decoding control in units ofsequences and decoding control in units of pictures. The first codestring analyzer 211 outputs residual encoding signals 261 which areacquired as a result of analyzing, to the prediction residual decoder212. Further, the first code string analyzer 211 outputs predictioninformation signals 265 which are acquired as a result of analyzing tothe prediction decoder 214. Still further, the first code stringanalyzer 211 outputs quantization value information which is acquired asa result of analyzing to the quantization value determiner 215.

2. Display Control Method

A method of analyzing the display control information in the displaycontrol information analyzer 202 and performing sort on the decodedpictures in an output order in the output picture setter 203 to be anoutput image will be described specifically with reference to the flowchart of the entire decoding process of FIG. 19.

First, the first code string analyzer 211-1 performs code stringanalyzing of header areas in units of sequences (S1901).

Next, the first code string analyzer 211-1 performs code stringanalyzing of header areas in units of pictures (S1902).

Next, the second code string analyzer 201-1 performs code stringanalyzing of extended information area (S1903). At this moment, thedisplay control information analyzer 202 acquires the display controlinformation in units of sequences and the display control information inunits of pictures.

Next, the picture data decoder 210-1 performs a series of decodingprocess to generate a decoded image of the objective pictures (S1904).Meanwhile, in step S1904, a common decoding process is applied withoutdepending on whether the video format is the progressive format or theinterlace format.

Next, the output picture setter 203 sorts the decoded image which isstored in the decoding order in a display order and selects a picture tobe displayed (S1905). At this moment, the output picture setter 203sorts the pictures according to a display control method acquired as aresult of analyzing by the display control information analyzer 202.

Next, when the processes on the currently processed picture to bedecoded are completed, the operation returns to step S1902 to proceed tothe decoding process of the next picture, and the processes from stepS1902 to step S1904 are repeated until the decoding processes of all ofthe pictures in the sequence are completed (S1905).

3. Sort of Pictures

As for the sort process of pictures in step S1905, the same processes asthose described with reference to FIGS. 11, 12, and 13 in the secondembodiment are performed. Therefore, the description of the sort processwill be omitted.

4. Configuration and Syntax of the Code String

Since the configuration and syntax of the code string to be decoded inthe embodiment are completely the same as those described in the firstembodiment with reference to FIGS. 6 to 8, the description of them willbe omitted here.

5. Summarization 5-1. Configuration

The video decoding device 200-1 of the embodiment decodes a code stringin units of pictures, the code string being acquired as a result ofencoding video signals in an interlace video format or a progressivevideo format in units of pictures. The video decoding device 200-1includes:

the second code string analyzer 201-1 which analyzes the code string toacquire an extended information area code string and a first codestring;

the second code string analyzer 201-1 (extended information areadecoder) which acquires display control information to be used indisplaying a decoded picture, from the extended information area codestring;

the picture data decoder 210-1 which decodes the first code string toacquire the picture by using common syntax analyzing method and a commonsignal processing method, which are not dependent on the video formats;and

-   -   the output picture setter 203 which, in the case where the        display control information indicates the progressive format,        sets the acquired picture as a frame and outputs the frame one        by one in a display order, and in the case where the display        control information indicates the interlace format, sets the        acquired picture as a field and outputs a pair of top field and        bottom field in a display order when the a pair of fields are        acquired, wherein the display control information includes        sequence unit display control information and picture unit        display control information, the sequence unit display control        information being commonly used in a display process of all        pictures which belong to a sequence to be decoded and the        picture unit display control information being individually used        in a display process of a picture to be decoded, and

the extended information area decoder 201-1 acquires each of thesequence unit display control information and the picture unit displaycontrol information from an extended information area in units ofpictures.

For example, each extended information area of each picture unit whichbelongs to the same sequence stores the sequence unit display controlinformation of the same value.

For example, the second code string analyzer 201-1 (extend informationarea decoder) acquires the sequence unit display control informationwhich includes a first identifier specifying whether the video signalsto be decoded are in the progressive format or the interlace format, and

the output picture setter 203 changes the output method of the picturebased on the acquired first identifier.

For example, in the case where the video signals to be decoded are inthe interlace format, the second code string analyzer 201-1 (extendinformation area decoder) acquires the sequence unit display controlinformation which includes a second identifier specifying whether twofields which belong to the same frame are always continual with eachother in the decoding order in the sequence, and

the output picture setter 203 changes the output method of the picturebased on the acquired second identifier.

For example, in the case where the video signals to be decoded are inthe interlace format, the second code string analyzer 201-1 (extendinformation area decoder) acquires the sequence unit display controlinformation which includes information indicating the maximum value ofan interval between the first field and the second field which belong tothe same frame in the decoding order in the sequence, and

the output picture setter 203 changes the output method of the picturebased on the acquired information indicating the maximum value.

For example, the second code string analyzer 201-1 (extend informationarea decoder) acquires the picture unit display control informationwhich includes a third identifier specifying that the picture to bedecoded is (1) to be displayed as a frame which belongs to theprogressive video signals, (2) to be displayed as a top field whichbelongs to the interlace video signals, (3) to be displayed as a bottomfield which belongs to the interlace video signals, or (4) to bedisplayed in another display format in the display process of thepicture to be decoded, and

the output picture setter 203 changes the output method of the picturebased on the acquired third identifier.

For example, the second code string analyzer 201-1 (extend informationarea decoder) acquires the picture unit display control informationwhich includes a third identifier specifying that the picture to bedecoded is (1) to be displayed as a top field which belongs to theinterlace video signals or (2) to be displayed as a bottom field whichbelongs to the interlace video signals, in the display process of thepicture to be decoded only in the case where the video signals to bedecoded are in the interlace format, and

the output picture setter 203 changes the output method of the picturebased on the acquired third identifier.

For example, the second code string analyzer 201-1 (extend informationarea decoder) acquires the picture unit display control informationwhich includes specification information specifying a picture that ispaired with the picture to be decoded and belongs to the same frame asthat of the picture to be decoded in the case where the picture to bedecoded is to be displayed as a top field or a bottom field whichbelongs to the interlace video signals, and

the output picture setter 203 outputs the picture to be decoded bysetting the picture to be decoded as paired with the picture specifiedin the specification information.

For example, the second code string analyzer 201-1 (extended informationarea decoder) acquires, as the specification information, display orderinformation which is assigned to the picture that is paired with thepicture to be decoded and belongs to the same frame as that of thepicture to be decoded.

For example, the second code string analyzer 201-1 (extended informationarea decoder) acquires, as the specification information, a differencevalue between the display order information which is assigned to thepicture to be decoded and the display order information which isassigned to the picture that is paired with the picture to be decodedand belongs to the same frame as that of the picture to be decoded.

A video decoding method of the embodiment decodes a code string in unitsof pictures, the code string being acquired as a result of encodingvideo signals in an interlace video format or a progressive video formatin units of pictures. The video decoding method comprises:

analyzing the code string to acquire an extended information area codestring and a first code string,

acquiring display control information to be used in displaying a decodedpicture, from the extended information area code string,

decoding the first code string to acquire the picture by using commonsyntax analyzing method and a common signal processing method, which arenot dependent on the video formats, and

in the case where the display control information indicates theprogressive format, setting the acquired picture as a frame andoutputting the frame one by one in a display order, and in the casewhere the display control information indicates the interlace format,setting the acquired picture as a field and outputting a pair of topfield and bottom field in a display order when the pair of top field andbottom field are acquired, wherein

the display control information includes sequence unit display controlinformation and picture unit display control information, the sequenceunit display control information being commonly used in a displayprocess of all pictures which belong to a sequence to be decoded and thepicture unit display control information being individually used in adisplay process of a picture to be decoded, and

each of the sequence unit display control information and the pictureunit display control information is acquired from an extendedinformation area in units of pictures.

5-2. Effects and the Like

With the video decoding device 200-1 according to the embodiment, adecoding process and display control are allowed to be performed onpicture data under a common control with the processing amount being notincreased for both a code string which is acquired as a result ofencoding a progressive video and a code string which is acquired as aresult of encoding an interlace video. That facilitates implementationof a decoding device.

As described in the third embodiment, in the present embodiment, theextended information area in which only the information unnecessary forthe encoding process (display control information, and the like) isdescribed and the other areas in which information necessary for thedecoding process is described are completely separated. As a result, thepresent disclosure facilitates sending of only the code strings neededby respective levels of the level of performing the decoding process andthe level of performing the display process to the respective levels forthe purpose like the above described use, therefore, improvesindependence of each level. That is, it allows to form the level ofperforming the decoding process (hardware) by using completely thecommon components in both the decoding device which supports theprogressive format and the decoding device which supports the interlaceformat.

Other Embodiments

Programs having the functions equivalent to the respective unitsincluded in the video encoding device and the video decoding devicedescribed in the above embodiments can be recorded in a recording mediumsuch as a flexible disk. That facilitates implementation of theprocesses described in the above embodiments in an independent computersystem. The recording medium is not limited to the flexible disk and maybe any medium as far as it can record a program, such as an opticaldisk, an IC card, and a ROM cassette.

The functions equivalent to the respective units included in the videoencoding device and the video decoding device described in the aboveembodiments can be realized as large scale integrated circuits (LSI).The large scale integrated circuits may be made into a chip whichincludes some or all of the functions. The LSI may be referred to as IC,system LSI, super LSI, and ultra LSI according to the integrationdensity.

Further, the technique of making the above described functions into anintegrated circuit is not limited to LSI, and the functions may beimplemented by a dedicated circuit or a general purpose processor. AnFPGA (Field Programmable Gate Array) which is capable of programing themanufactured LSI or a reconfigurable processor which is capable ofreconfiguring connection and setting of the circuit cells inside the LSImay be used.

Furthermore, when a new technology of an integrated circuit will bedeveloped to replace the LSI and the like as a result of advancement ofthe semiconductor technology or another derivative technology, it is amatter of course that the new technology may be used in making thefunctional blocks into an integrated circuit.

The present disclosure may be applied to a broadcast wave recordingapparatus including the above described video encoding device and videodecoding device, such as a DVD recorder and a BD recorder, whichcompresses and records the broadcast waves sent from a broadcaststation.

At least some of the functions of the video encoding device and thevideo decoding device according to the above described embodiments ortheir modifications may be combined together.

INDUSTRIAL APPLICABILITY

The present disclosure is useful as a video encoding device whichencodes each picture of an input video and outputs the encoded videodata or a video decoding device which decodes the encoded video data togenerate a decoded image in a video camera, a digital camera, a videorecorder, a mobile phone, and a personal computer, for example.

What is claimed is:
 1. A decoding method for decoding an encoded bitstream which is generated by encoding video in units of pictures,comprising: acquiring video format information indicating whether thevideo is encoded with a video format among an interlace format or aprogressive format, from a header of a sequence which is a unit of thevideo, (i) setting each of all frames or all fields which are includedin the video, as a picture, regardless of whether the video format isthe interlace format or the progressive format, and (ii) setting a POCindicating display order to each of all of the set pictures one by one,the POC being different each other, and decoding a picture to be decodedwhich is the frame or the field with reference to a picture previouslydecoded before decoding the picture to be decoded, wherein in thedecoding, the video is decoded with a syntax structure which is notdependent on the video format.
 2. The decoding method according to claim1 further comprising, further acquiring display parameter indicatingdisplay format of the pictures to be decoded, from an extendedinformation area in units of pictures, the display format being setcorresponding to the video format information, determining whether thepictures to be displayed which has already decoded is to be displayed asa frame or a field based on the display parameter.
 3. A decoding devicefor decoding an encoded bit stream which is generated by encoding videoin units of pictures, comprising: an acquirer that acquirers videoformat information indicating whether the video is encoded with a videoformat among an interlace format or a progressive format, from a headerof a sequence which is a unit of the video, a picture setter that (i)each of all frames or all fields which are included in the video, as apicture, regardless of whether the video format is the interlace formator the progressive format, and (ii) sets a POC indicating display orderto each of all of the set pictures one by one, the POC being differenteach other, and a decoder that decodes a picture to be decoded which isthe frame or the field with reference to a picture previously decodedbefore decoding the picture to be decoded, wherein in the decoder, thevideo is decoded with a syntax structure which is not dependent on thevideo format.
 4. The decoding device according to claim 3, the decoderdecodes a top field and a bottom field belonging to the same fieldcontinually in the decoding order, when the video format is theinterlace format.
 5. The decoding device according to claim 3, theacquirer acquirers display parameter indicating display format of thepictures to be decoded, from an extended information area in units ofpictures, the display format being set corresponding to the video formatinformation, wherein the decoding device further comprising, a displaydeterminer that determines whether the pictures to be displayed whichhas already decoded is to be displayed as a frame or a field based onthe display parameter.
 6. The decoding device according to claim 4, theacquirer acquirers display parameter indicating display format of thepictures to be decoded, from an extended information area in units ofpictures, the display format being set corresponding to the video formatinformation, wherein the decoding device further comprising, a displaydeterminer that determines whether the pictures to be displayed whichhas already decoded is to be displayed as a frame or a field based onthe display parameter.